Final Report : Power Price Prediction#

Contributors#

  • Arjun Radhakrishnan

  • Sneha Sunil

  • Gaoxiang Wang

  • Mehdi Naji

Executive Summary#

Numerous Alberta-based organizations rely heavily on energy to fuel their business operations and are in search of an effective forecasting tool that offers accurate and interpretable predictions. Our business solution precisely addresses this need by offering an interpretable and explainable data science product deployed on the cloud, specifically designed for power price prediction in the Alberta Energy Market. Our solution equips organizations with the ability to make knowledgeable decisions about their energy purchases by forecasting hourly energy prices for the next 12 hours, supplemented with confidence intervals. It is noteworthy that our forecasting system has demonstrated a 23.21% improvement in prediction accuracy compared to the current system, which only forecasts for the next 6 hours. Our solution also addresses the lack of interpretability and explainability of predictions, which is showcased in an intuitive Tableau dashboard.

Introduction#

Over the past few decades, the electricity market in Alberta province has undergone a significant transformation, shifting from regulated to competitive and deregulated environments where prices are determined by the interplay of supply and demand dynamics in a competitive marketplace. Prices are influenced by various participants in the market, including power generators, transmission companies, and retailers. Consequently, the deregulated nature of the market plays a crucial role in the volatility of power prices. To illustrate the extent of this volatility, an interactive plot below showcases the substantial variations in price where we can witness the dramatic ups and downs. This serves as a reminder of the inherent complexity of the data science problem that needs to be addressed.

<<<<<<< HEAD
<<<<<<< HEAD

Energy-intensive industries in Alberta heavily depend on accurate price predictions to plan for future energy costs and optimize their operations. In light of the growing uncertainty caused by escalating price volatility, the significance of accurate electricity price predictions cannot be emphasized enough. These predictions play a pivotal role in enabling stakeholders, including energy buyers, to navigate the market successfully by efficiently strategizing their operations. Currently, these organizations depend on energy forecasting tool published by AESO - Alberta Electric System Operator to determine their energy costs in advance for their business operations. AESO is an independent system operator in Alberta responsible for operating the electrical grid, facilitating the competitive electricity market, and managing the entire power distribution system for the province. The power pool price for every hour is finalized by AESO based on supply and demand. However, the current energy forecasts published by AESO only provide a short-term coverage of 6 hours into the future, which is volatile and lacks interpretation or model visibility.

To reduce their expenses, companies could plan and potentially explore alternative energy options if they have access to accurate forecasts which covers a longer window and is also interpretable and explainable. To address the challenges faced by organizations in Alberta, our product offers a comprehensive solution that empowers them to effectively analyze costs, optimize energy purchases, and ultimately maximize their profits. The scientific objectives of our product are

  • Forecasting energy prices for the next 12 hours

  • Interpretability and explainability of predictions

  • Scalable and real-time prediction pipeline

  • Reactive Tableau Dashboard for visualizing the real-time results

Data Science Techniques#

Our project utilized two primary data sources:

Open-source Tableau Data: We had access to historical hourly time series data published by AESO in Tableau.

Open-source API Data: AESO provides a public API service that grants us access to real-time and historical data for selective features.

Our initial dataset consisted of approximately 110 features and ~ 72,000 rows, covering various aspects such as power generation/supply by fuel type, energy load distribution across regions, import and export data, and system marginal price data. The primary target we focused on forecasting was the power pool price (CAD). This price represents the balanced average power price per hour for the Alberta province as determined by AESO. It is capped between 0 and 1000 to ensure that the Alberta electricity market is stable and fair. Our feature set predominantly comprises numerical data, with one exception of an ordinal feature that was engineered by us.

<<<<<<< HEAD

Examining the plots reveals consistent trends in energy prices. On weekdays, prices are higher during peak hours (8 am to 8 pm) due to increased demand from business operations. Weekdays also show lower prices during off-working hours. However, weekends have a different pattern, with higher prices in the evenings. These observations are supported by the autocorrelation function plots, which clearly demonstrate daily seasonality in energy prices. To capture the combined effects of day of the week and peak/off-peak hours, an ordinal feature called ‘weekly_profile’ was created to represent time-related variables and energy pricing dynamics.

In the below plot, we can clearly see a daily seasonality in this plot (for every 24 hours)
Note : ACF plot depicts the correlation between the price and its lagged values

<<<<<<< HEAD

Data preprocessing#

For the modeling purpose, we partitioned our data into two subsets: the training set from January 2021 to January 2023 and the testing set for February 2023. Given the absence of real-time data for all features through the API, we leveraged historical data to simulate a real-time prediction system. When real-time data becomes accessible for the clients, they can seamlessly swap the data sources, thereby enabling real-time data flow into the model which will make real-time predictions.

In our pursuit to accurately predict future prices based on historical values of influential factors, we transformed the time-series data of all the features into a tabular format by creating lagged versions of both the target variable and the relevant features. Each lagged version corresponds to a specific hour, with the price for that hour being used as the target variable.

Feature Selection and Engineering#

In the process of feature selection out of 110 features, our primary strategy involved examining the correlations between various features and the price. Considering the importance of interpretability in our model, we also conducted comprehensive market research and engineered several key features showing a significant correlation with the price. One such feature is the gas reserve margin, a buffer/reserve of gas, readily available to generate electricity to meet sudden load demands and peak hours. As evidenced in our data visualizations, a dwindling gas reserve tends to correspond with an increase in price. In the case of gas supply mix which is the proportion of energy generation using gas by the total energy generation, when the supply is mostly using gas, the price increases as gas is costly compared to the rest of the sources.

For more information about the key engineered features, please check out the Key Engineered features section on the left.

<<<<<<< HEAD

We further refined our feature set by leveraging the coefficients from training an Elastic Net CV model and the feature importances deduced from training a Random Forest Regressor model.

Pursuing a second strategy, we investigated the correlation between lagged features and future prices projected for periods ranging from 1 to 12 hours. We identified features exhibiting correlations of absolute value greater than 0.3 and incorporated them into our feature set. Interestingly, both strategies resulted in almost identical sets of features.

<<<<<<< HEAD

Modelling Strategy#

Since the price is volatile, tackling our first scientific objective which is forecasting energy for the next 12 hours seemed like a complex task. Hence, we needed models that are apt for time series forecasting, which can pick up temporal patterns. As a baseline model for our problem, we chose the SARIMA (Seasonal Autoregressive Integrated Moving Average) model for univariate forecasting as it is a popular classical time series forecasting technique that incorporates both autoregressive and moving average components along with seasonality. It also supports probabilistic forecasting and the generation of confidence intervals.

However, recognizing that our problem involved multi-step forecasting, we decided to transition to more sophisticated machine-learning models to improve the accuracy of our forecasts. In this approach, we incorporated the key engineered features in addition to using past price values.

For our forecasting horizon of 12 steps, we implemented the direct strategy, which involved training 12 individual models using the same historical data until the cut-off point, each responsible for predicting the price for a specific timestep within the horizon (1, 2, 3… 12). The cut-off hour refers to the specific point in time up to which you use the data to train your model. In this approach, Model 1 would consistently predict the power price for the next timestep, while Model 12 would forecast the price for 12 timesteps into the future. By adopting this approach, we avoided the accumulation of errors and controlled error propagation that can occur in the recursive strategy. Additionally, when a new data point became available, we updated the cut-off time by 1 hour and retrained all 12 models using the most recent data. This allowed us to continuously incorporate real-time data and ensure reliable future predictions. To build our base pipeline, we leveraged the sktime package, a widely-used package that supports time series forecasting.

Cross Validation Split#

Our training data spanned from January 1st 2021, to January 31st 2023. To capture any seasonality in price variation over time, we initially trained our models using two years’ worth of data. For cross-validation, we utilized the entire month of January 2023, creating 63 folds to validate our models. Since our data is time series data, preserving the temporal order of the data in each fold was crucial. The first fold consisted of an initial training window of two years and a validation set spanning 12 hours of data. We made predictions for these 12 hours and compared them with the actual prices in the validation set to calculate the errors. We then expanded the training window by including the 12 hours of data from the validation set and proceeded to predict the next 12 hours. This process was repeated for a total of 63 folds.

Experimented models#

Our initial choice for modeling was Elastic Net CV, a linear regression model that effectively handles multicollinearity and supports feature selection. Given our emphasis on interpretability, a linear model was a suitable option as it offers a straightforward interpretation of coefficients and relationships between variables. In addition to Elastic Net CV, we also considered XGBoost and Light GBM which are powerful gradient-boosting algorithms that excel in efficiency, accuracy, handling large datasets, and managing multicollinearity. LightGBM specifically satisfied all our requirements, including scalability and efficient model fitting. It particularly excelled when loaded on a GPU, enabling faster training times for large datasets. Notably, LightGBM supports warm initialization, enabling rapid model refitting on new data, which was essential for our real-time forecasting scenario.

Evaluation Metric#

To assess model performance, we selected the Root Mean Square Error (RMSE) as our evaluation metric because it aligns with our project’s focus on interpretability and is easily understandable as the error value is in the same scale as the actual values measured in CAD. We are using two variations of RMSE as evaluation metrics in our project. For each hour, we generated 12-step predictions into the future. Then for each n step prediction (predictions made n step into the future), we calculated the RMSE with the actual value at that hour, which is the step wise error. For eg, we compared our 1-step predictions with the corresponding actual values to obtain Step 1 error. The second one is average error per prediction, where for each prediction instance that has 12 forecasts, we calculate the RMSE and average it.

Cross Validation Results#

Model Avg RMSE(CAD) RMSE SD(CAD)
0 LightGBM 82.962592 79.157233
1 ARIMA 88.812002 67.579395
2 Elastic Net 89.302110 66.732290
3 XGBoost 94.000088 70.299748

The table presents the average and standard deviation of RMSEs calculated across 63 folds during cross-validation for the specified models.

As seen from the above results, all the models demonstrated good performance according to the RMSE metric, but Light GBM excelled, considering our specific needs. Its computational efficiency, rapid fit times, and warm state capability - which retains and updates previously learned data - made it ideal for quick model updates when AESO published new data. Additionally, Light GBM captured spikes in prices, whereas the other models struggled to capture such sudden changes, resulting in flatter predictions. Light GBM also supported quantile regression, enabling us to generate confidence intervals for our predictions. All in all, its superior performance and adaptability made Light GBM the best out of the experimented models for our forecasting pipeline.

Using sktime, we encountered a challenge where refitting the data to all 12 Light GBM models became time-consuming. To address this problem, we implemented custom code that significantly reduced the refit times from approximately 4.5 minutes to less than 0.5 seconds. Despite a slight increase in our RMSE, this optimization was a significant achievement for us, allowing us to overcome the computational burden and enhance the efficiency of our pipeline.

Interpretibility of predictions#

To achieve our second scientific objective of obtaining feature importance and interpreting the predictions of our model, we relied on the SHAP (SHapley Additive exPlanations) framework. SHAP enables us to explain and interpret the individual predictions made by our model. For each prediction the model makes, we can easily obtain the SHAP values of each of the features for this prediction, which will quantify the impact and contribution of each feature for the prediction. To quantify the uncertainty of our predictions, we utilized quantile regression to obtain the 95% confidence intervals.

For more information, please check the Appendix section.

Test Results#

Coming to the test results, our modeling pipeline outperformed AESO’s model across all timesteps as shown in the table. Our prediction pipeline demonstrates superiority not just in short-term 1-step and 2-step forecasts but continues to maintain lower RMSE values right up to 12-step forecasts. Contrary to the increasing RMSE as we forecast for farther timesteps, it is important to highlight that our model consistently demonstrates a comparatively lower range of RMSE compared to AESO, even during higher step predictions. This implies that our model’s accuracy doesn’t degrade significantly with the extension of the forecast horizon, which is a highly desirable feature in multi-step forecasting models.

We can see below the stepwise errors for the time range of May 26 - May 30 (5 days). Our model predictions are better than AESO predictions by ~23.21% even while predicting twice the number of steps into the future. ​

Comparison of our Step wise RMSE with AESO#

The fit time for our pipeline is 16.063 mins and the prediction time is 6.273 mins
1 Step RMSE 2 Step RMSE 3 Step RMSE 4 Step RMSE 5 Step RMSE 6 Step RMSE 7 Step RMSE 8 Step RMSE 9 Step RMSE 10 Step RMSE 11 Step RMSE 12 Step RMSE Avg Step RMSE
0 104.79 116.36 143.97 152.41 191.53 217.49 NaN NaN NaN NaN NaN NaN 154.42
1 102.52 114.55 121.77 122.37 126.82 136.51 124.75 122.43 114.38 111.14 114.85 110.57 118.56

Average Error per prediction for test set#

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
File ~\miniconda3\envs\slalomenv\Lib\site-packages\pandas\core\indexes\base.py:3802, in Index.get_loc(self, key, method, tolerance)
   3801 try:
-> 3802     return self._engine.get_loc(casted_key)
   3803 except KeyError as err:

File ~\miniconda3\envs\slalomenv\Lib\site-packages\pandas\_libs\index.pyx:138, in pandas._libs.index.IndexEngine.get_loc()

File ~\miniconda3\envs\slalomenv\Lib\site-packages\pandas\_libs\index.pyx:165, in pandas._libs.index.IndexEngine.get_loc()

File pandas\_libs\hashtable_class_helper.pxi:5745, in pandas._libs.hashtable.PyObjectHashTable.get_item()

File pandas\_libs\hashtable_class_helper.pxi:5753, in pandas._libs.hashtable.PyObjectHashTable.get_item()

KeyError: 'model_start_date'

The above exception was the direct cause of the following exception:

KeyError                                  Traceback (most recent call last)
Cell In[14], line 7
      5 results = results.reset_index(drop=True)
      6 # Convert string to datetime
----> 7 results['model_start_date'] = pd.to_datetime(results['model_start_date'])
      8 results['prediction_end_date'] = pd.to_datetime(results['prediction_end_date'])
      9 results

File ~\miniconda3\envs\slalomenv\Lib\site-packages\pandas\core\frame.py:3807, in DataFrame.__getitem__(self, key)
   3805 if self.columns.nlevels > 1:
   3806     return self._getitem_multilevel(key)
-> 3807 indexer = self.columns.get_loc(key)
   3808 if is_integer(indexer):
   3809     indexer = [indexer]

File ~\miniconda3\envs\slalomenv\Lib\site-packages\pandas\core\indexes\base.py:3804, in Index.get_loc(self, key, method, tolerance)
   3802     return self._engine.get_loc(casted_key)
   3803 except KeyError as err:
-> 3804     raise KeyError(key) from err
   3805 except TypeError:
   3806     # If we have a listlike key, _check_indexing_error will raise
   3807     #  InvalidIndexError. Otherwise we fall through and re-raise
   3808     #  the TypeError.
   3809     self._check_indexing_error(key)

KeyError: 'model_start_date'
<<<<<<< HEAD
=======
>>>>>>> main